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Abstract: 


Creating  models  and  experiment  platforms  that  study  the  relationships  and  interactions  between 
composite  networks  will  enable  the  design  of  tactical  networks  to  maximize  mission 
performance  of  an  organization.  Tactical  environments  can  be  conceptualized  by  complex 
networks,  a  composition  of  communication,  social/cognitive,  and  information  networks.  We 
describe  an  approach  to  integrate  ELICIT  (Experimental  Laboratory  for  Investigating 
Collaboration,  Information-sharing  and  Trust)  with  an  existing  wireless  emulation  test  bed,  the 
Wireless  Emulation  Laboratory  (WEL).  These  modifications  enhance  the  capability  to  simulate 
more  realistic  communication  models  during  trials  of  ELICIT  experiments.  Gaining  an 
understanding  of  how  communication  network  parameters  affect  decision-making  performance 
in  social  networks  will  enable  the  design  of  tactical  networks  to  optimally  support  mission 
operations.  Experiments  using  the  emulation  environment  will  provide  validation  to  the 
parameters  in  ELICIT  used  to  simulate  communication  network  parameters.  Additionally,  we 
study  how  specific  ELICIT  parameters  can  be  used  to  represent  both  social  and  communication 
network  concepts. 
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Introduction 


Tactical  networks  are  comprised  of  many  elements  which  interact  dynamically  to  support 
Warfighter  mission  performance.  Typically  considered  to  include  all  military  assets  available  to 
the  Warfighter,  the  scope  of  the  tactical  scenario  has  expanded  to  include  elements  from  other 
domains  (i.e.  culture,  society,  politics,  and  psychology).  Tactical  environments  began  to  consider 
these  elements  to  support  Warfighter  interactions  with  secondary  groups  such  as  local  nationals 
and  non-governmental  agencies.  Organizations  such  as  the  Department  of  Defense  and  the 
Department  of  Homeland  Security  have  proposed  strategies  to  deal  with  the  increasing  scope  of 
their  tactical  situations.  Information  sharing  is  a  crucial  element  of  their  organizational  capability 
[1-2].  Each  of  these  organizations  recognizes  the  complexity  of  their  situations  calls  for  the 
consideration  of  networks  with  multidisciplinary  elements  such  as  culture,  policies,  coalition 
members,  and  technology.  In  many  of  these  cases,  the  performance  of  the  Warfighter  is  the 
primary  interest.  Understanding  and  being  able  to  characterize  the  interactions  between 
multidisciplinary  networks  will  enable  network  design  strategies  to  optimize  Warfighter  mission 
performance.  A  key  tool  for  developing  this  understanding  is  the  existence  of  a  test  bed  that 
models  all  the  characteristics  of  complex  networks  and  can  be  used  for  research  experiments. 

The  goal  of  this  work  is  to  improve  the  understanding  of  complex  networks  and  provide  a  viable 
test  bed  to  verify  cross-layer  network  models.  This  paper  first  describes  a  command  and  control 
experiment  platform  called  ELICIT  (Experimental  Laboratory  for  Investigating  Collaboration, 
Information-sharing,  and  Trust)  [18].  This  platform  has  been  used  to  model  both  organizational 
behavior  constructs  as  well  as  communication  networks.  We  use  this  platform  to  study  the  scope 
and  range  of  parameters  of  complex  networks  involving  communication  networks  and 
social/cognitive  networks.  We  then  describe  current  work  that  has  improved  the  communication 
network  modeling  capability  within  the  ELICIT  platform.  This  involves  the  integration  of 
ELICIT  with  a  communications  emulation  platform  called  The  Wireless  Emulation  Laboratory 
(WEL).  Lastly,  we  present  results  to  illustrate  the  increased  fidelity  of  the  communication 
models  used  in  ELICIT  experiments. 

Complex  Networks 

Tactical  networks  can  be  represented  and  modeled  through  complex  networks.  Network  science 
[3]  is  a  research  area  intended  to  increase  the  understanding  of  complex  networks,  that  is, 
networks  that  include  communication,  information  and  social/cognitive  (CIS)  networks.  One  of 
the  major  objectives  of  this  field  of  study  is  to  increase  the  understanding  of  the  interactions 
between  the  constituent  networks  of  the  complex  network.  The  majority  of  network  research 
considers  one  of  the  constituent  networks,  while  a  characterization  of  the  entire  complex  network 
may  provide  a  more  accurate  representation  of  the  increasingly  complex  scope  of  tactical 
networks.  Figure  1  illustrates  the  concept  of  complex  networks  being  comprised  of 
social/cognitive,  information  and  communication  networks. 
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Figure  1.  Complex  network  representation  of  CIS  Networks 

Studies  considering  the  performance  of  communication  networks  tend  to  establish  analytical 
measures  of  technical  metrics  (i.e.  bandwidth,  latency,  throughput,  or  packet  delivery  rates)  [4- 
8].  These  studies  do  not  emphasize  the  impact  of  the  stakeholder.  In  tactical  networks  the 
stakeholder  is  the  Warfighter.  A  naive  approach  to  maximizing  mission  performance  in 
communication  networks  is  to  provide  full  bandwidth  and  full  resolution  to  all  users  all  the  time. 
From  a  communication  resources  perspective,  this  approach  is  extremely  inefficient. 
Communications  overhead  must  also  be  taken  into  consideration.  Strategies  for  the  design  of 
tactical  networks  must  consider  the  efficiency  of  communication  network  resources. 

Social/cognitive  networks  research  has  generated  numerous  qualitative  studies  by  developing 
models  that  aim  to  represent  cognitive  processes  and  social  phenomenon  [9-14],  In  these  studies, 
the  communication  models  that  are  employed  tend  to  be  over-simplified.  To  date,  much  of  the 
work  has  emphasized  qualitative  over  quantitative  data.  Further,  the  studies  involving  human 
subjects  are  difficult  to  conduct,  and  it  is  even  more  difficult  to  gather  accurate  metrics  and 
measurements  during  these  experiments.  The  subjective  nature  of  measurements  and  procedures 
used  to  gather  this  data  often  limits  the  significance  of  these  studies.  The  generation  of 
quantitative  models  that  use  software  agents  rather  than  humans  will  enable  analysis  of  complex 
networks  across  multiple  domains.  Strategies  for  the  design  of  tactical  networks  must  consider 
the  efficiency  of  the  decision-making  ability  of  the  Warfighter. 

The  modeling  of  these  networks  may  result  in  the  emergence  of  tradeoffs  between  human 
(mission)  performance  metrics  (social/cognitive  network  metrics)  and  the  communications 
overhead  (communication  network).  The  Army  has  established  a  network  science  collaborative 
technology  alliance  which  is  a  collaboration  between  government,  industry  and  academic 
institutions  to  perform  foundational,  cross-cutting  research  for  a  fundamental  understanding  of 
interactions,  interdependencies,  and  common  underlying  science  among  social/cognitive, 


information,  and  communication  networks.  The  aim  of  network  science  is  to  model  the 
interactions  of  complex  networks  which  include  communication,  information  and 
social/cognitive  networks  [3].  A  goal  of  the  Army  is  to  maximize  the  mission  performance  of  the 
Warfighter.  By  using  concepts  of  network  science  to  represent  tactical  environments,  it  is 
possible  to  develop  cross-network  models  to  enable  the  prediction  and  analysis  of  the 
performance  of  the  Warfighter  in  these  situations. 

In  command  and  control  (C2)  research,  complex  endeavors  are  defined  as  situations  that  are 
“characterized  by  complex,  multi-dimensional  effects  spaces”  [15].  The  study  of  complex 
endeavors  enable  military  planning  approaches  by  considering  the  interactions  of  the  military 
units  with  coalition  military  partners  and  other  non-military  organizations.  These  interactions  are 
dynamic  and  unpredictable  in  nature.  The  C2  maturity  model  /  approach  space  was  proposed 
[16]  to  represent  the  state  and  capability  of  organizations.  The  approach  space  is  defined  by  a 
function  of  three  non-orthogonal  axes/variables:  allocation  of  decision  rights,  distribution  of 
information  and  patterns  of  interaction.  Five  organizational  operating  states  are  then  mapped  to 
this  three-dimensional  C2  approach  space:  conflicted,  deconflicted,  coordinated,  collaborative, 
and  edge.  The  C2  approach  space  is  shown  in  Figure  2.  Based  on  the  mission,  one  of  the  five 
operating  states  may  be  optimal  performance-wise.  It  is  not  necessarily  the  case  that  the  edge 
organization  is  universally  optimal. 
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Figure  2.  C2  Approach  Space  (CCRP) 

To  apply  the  C2  approach  space  to  the  dynamics  of  tactical  networks,  the  concept  of  agility  in 
organizations  was  proposed  as  a  measure  of  how  to  adapt  to  a  given  network  state  [15].  Rather 
than  attempting  to  predict  network  conditions  and  determine  which  C2  approach  space  would 
optimally  handle  the  mission  given  the  network  state,  it  is  proposed  that  the  more  effective 


network  is  one  that  is  able  to  successfully  cope  with  changes  in  the  environment  (system).  The 
organization  would  effectively  handle  degradation/loss  of  capability,  change  in  objectives,  and 
change  in  the  conditions  of  the  coalition/individuals.  The  goal  of  agility  is  to  maintain  acceptable 
performance,  effectiveness  and  efficiency  (thus  requiring  timely  recognition,  appropriate 
response,  and  timely  response).  The  concept  of  agility  can  be  applied  to  people,  C2  approaches, 
organizations,  processes,  systems,  architectures,  policy,  doctrine,  tactics,  and  acquisition  [17].To 
establish  an  abstract  model  that  combines  network  science  and  concepts  of  command  and  control 
research,  the  parameters  of  communication,  information,  and  social/cognitive  networks  can  be 
combined  to  describe  the  network  effects  complex  endeavors  encompass.  In  this  model,  C2 
Maturity  is  a  set  of  operating  states  of  a  social/cognitive  network,  and  C2  agility  is  the  ability  of 
a  complex  network  to  adapt.  Network  science  research  is  currently  focused  on  creating  models  of 
these  interactions  and  has  not  necessarily  identified  specific  strategies  to  demonstrate  or 
implement  agility  within  complex  networks. 

The  joint  consideration  of  network  science  and  command  and  control  research  concepts 
inevitably  brings  together  concepts  from  multiple  disciplines  which  do  not  have  a  shared 
vocabulary.  In  addition,  there  are  common  parameters  in  each  of  the  CIS  networks  of  the 
complex  network  approach  that  can  have  different  range  and  scope  of  value.  For  instance,  a 
processing  delay  in  communication  networks  may  have  a  scope  of  10-100  ms  while  the 
analogous  delay  in  social  networks  may  have  a  range  of  10-20  s.  This  mismatch  in  representation 
may  dilute  the  sensitivity  of  experimental  results  with  respect  to  the  common  parameter.  Being 
able  to  individually  express  these  parameters  in  a  test  environment  may  result  in  the  emergence 
of  tradeoffs  or  optimal  operating  values  for  these  parameters  within  CIS  networks. 

ELICIT  Platform 

ELICIT  was  developed  by  the  United  States  Department  of  Defense  Command  and  Control 
Research  Program  (CCRP)  of  the  Office  of  the  Assistant  Secretary  of  Defense  for  Networks  and 
Information  Integration  (OASD/NII)  as  part  of  their  initiative  to  develop  and  test  principles  of 
organization  that  enable  transformation  from  traditional  hierarchy-based  command  and  control 
practices  toward  more  agile  practices  that  transfer  more  power  and  decision  rights  to  the  edge  of 
the  organization  [19].  Considerable  research  has  been  conducted  to  date  using  ELICIT  including 
[21-23], 

This  ELICIT  platform  has  configurable  scenarios  that  enable  groups  to  focus  on  the  task  of 
identifying  the  “who”,  “what”,  “where”,  and  “when”  of  a  fictional  insurgent  threat.  Specific 
packets  of  partial  information  in  the  form  of  “factoids”  are  provided  periodically  to  each  of  the 
participants  during  an  experiment  session.  The  factoids  and  their  distribution  are  structured  so 
that  no  one  participant  receives  all  the  information  necessary  to  perform  the  task;  thus, 
information  sharing  is  required  in  order  for  any  participant  to  be  able  to  determine  a  solution  to 
the  ELICIT  problem. 

A  web-based  ELICIT  session  starts  with  the  factoids  being  randomly  seeded  to  the  participants 
in  the  trial.  Participants  must  then  share  and  receive  this  information  with  other  participants  to 
gain  knowledge  of  the  threat.  Additionally,  participants  may  have  access  to  a  set  of  centralized 
information  sources  called  websites  that  allow  participants  either  to  post  or  pull  factoids  from 


these  websites.  Periodically,  the  participants  can  send  their  knowledge  of  the  threat  to  the 
ELICIT  administrative  system.  ELICIT  provides  an  instrumented  task  environment  that  captures 
and  time  stamps  participants’  information  sharing  activities. 

To  measure  their  performance,  we  use  a  measure  called  correctness.  Correctness  is  a  measure 
that  represents  the  level  of  situational  awareness  within  this  scenario.  Correctness  is  measured  by 
the  accuracy  of  the  participant’s  knowledge  of  the  “who”,  “what”,  “where”,  and  “when”  and  has 
a  scale  from  0  to  1.  The  details  “who”,  “what”,  and  “where”  are  scored  with  0  or  1,  and  “when” 
has  a  score  of  {0,  0.25,  0.5,  0.75  and  1.0},  allowing  for  partial  correctness.  The  overall 
correctness  score,  C: 


C  =  0.25  (  WHO  +  WHERE  +  WHAT  +  WHEN  ). 

Sensemaking  agents  were  developed  for  use  in  ELICIT  (abELICIT)  [24,  25],  enabling 
campaigns  of  experiments  involving  agents,  either  in  lieu  of  or  in  conjunction  with  human 
participants.  The  sensemaking  agent  software  instantiates  semi-intelligent,  configurable  agents 
that  are  designed  to  behave  like  humans.  These  agents  are  able  to  take  the  place  of  human 
participants  in  ELICIT  experiments,  and  form  a  mental  model  of  the  information  in  the  factoids 
received  and  of  the  members  of  the  group  in  which  they  operate.  That  is,  as  an  agent  participates 
in  an  experiment  it  generates  scenario  specific  situational  awareness  that  can  be  drawn  upon  to 
make  decisions  about  behavior.  The  sensemaking  agent  is  able  to  formulate  ELICIT  messages 
based  on  awareness  and  understanding  of  the  factoids  to  which  it  has  been  exposed. 

In  addition  to  having  this  awareness  or  sensemaking  capability,  the  agents  have  additional 
configurable  variables  to  define  their  personalities,  capabilities  and  styles  of  social  interaction 
with  the  other  experiment  participants.  Using  these  variables,  agents  can  be  configured  to  operate 
in  human  timeframes,  show  human  levels  of  variability  and  human  personality  traits  such  as 
tendencies  to  hoard  information,  reciprocate  favors  and  trust  team  members  (among  others). 

In  terms  of  architecture,  Figure  3  delineates  a  high-level  view  of  the  sensemaking  agent  logic 
flow.  The  sense  making  agent  explicitly  models  the  major  mental  steps  that  humans  take  when 
performing  ELICIT  tasks.  The  model  accounts  for  people  taking  these  steps  in  differing  orders 
and  in  varying  time  frames.  Interested  readers  are  directed  to  [24]  for  details. 


Figure  3.  High-Level  View  of  Sensemaking  Agent  Logic  Flow. 


It  is  important  to  note  that  the  sensemaking  agent  behaviors  have  been  validated  and  calibrated  in 
part  through  comparison  with  human  behaviors  in  numerous  ELICIT  experiments.  Indeed, 
previous  ELICIT  based  experiments  provide  a  rich  set  of  data  on  which  to  model  agent  behavior. 
These  data  include  transaction  logs  that  record  and  timestamp  every  action  taken  by  each 
participant,  scratch  paper  used  by  experiment  participants,  and  information  gleaned  from 
participants  in  post  experiment  discussion  sessions.  This  information  and  experience  provide  a 
strong  basis  for  modeling  human  information-sharing  and  information-processing  behaviors  and 
illuminate  insight  into  the  mental  models  used  by  people  when  participating  in  the  experiments. 
The  abELICIT  implementation  includes  about  50  configurable  parameters  that  can  be  used  to 
tailor  the  sensemaking  agents’  behavior.  Thus,  ELICIT  is  able  to  model  the  social  cognitive 
elements  of  human  decision  making  in  various  network  configurations.  Its  ability  to  model  the 
dynamics  of  communication  networks  is  limited. 


Multi-disciplinary  concepts 

Studies  in  network  science  /  complex  endeavors  involve  concepts  from  multiple  disciplines. 
Difficulty  in  creating  cross  or  multi-disciplinary  models  is  caused  in  part  by  non-shared 
vocabularies.  Models  of  complex  networks  will  need  to  distinguish  between  similar  concepts  in 
different  network  layers.  For  example,  transmission  delays  in  communication  and  social 
networks  should  not  be  represented  by  a  single  delay  parameter.  The  goal  of  this  work  within 
network  science  and  C2  agility  research  is  to  develop  models  and  strategies  that  will  maintain 
acceptable  performance  in  the  presence  of  the  set  of  expected  network  conditions  spanning  the 
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entire  complex  network  space.  A  more  specific  goal  of  this  paper  is  to  develop  a  comprehensive 
network  model  that  bridges  social/cognitive  network  and  communication  network  models.  Our 
approach  is  to  survey  a  set  of  parameters  common  to  each  of  the  constituent  network  models  in 
complex  networks.  We  perform  this  survey  by  using  parameters  found  in  the  agents  designed  for 
the  ELICIT  experiment  platform. 

In  related  work,  Nissen  and  Alberts  [27]  connect  the  C2  maturity  and  agility  models  to  concepts 
in  organization  and  management  theory.  Connections  are  drawn  between  agility  and  contingency 
theory.  C2  maturity  models  are  compared  to  organizational  archetypes.  Parallels  between 
organizational  design  factors  and  C2  field  concepts  are  drawn.  The  outcome  of  [27]  is  a 
preliminary  work  providing  a  “Rosetta  Stone”  between  C2  and  organizational  management 
theory.  A  method  to  jointly  characterize  organizations  using  C2  and  organizational  management 
theory  concepts  is  also  proposed. 

When  establishing  connections  between  social/cognitive  networks  and  communication  networks, 
existing  work  tends  to  evaluate  social  networks  and  communications  between  entities  using 
social  network  analysis  tools  that  make  over  simplified  assumptions  with  regard  to 
communications.  Analysis  of  social  structures  typically  employs  sociograms,  which  can  illustrate 
interactions  or  the  volume  of  interaction  between  nodes  in  the  network.  Sociograms  can  be 
established  by  measurement  of  email  communications  [14,  28].  However,  in  these  studies,  the 
quality  of  the  communications  links  is  not  considered.  Email  interaction  provides  a  coarse 
representation  of  the  interaction  as  well  as  the  quality  of  the  network.  Sociograms  can  be  used  to 
study  social  network  analysis  metrics  such  as  centrality,  betweenness,  and  social  tie  strength. 
Studying  the  interactions  represented  in  sociograms  and  the  quality  of  the  interactions  by 
investigating  the  dynamics  of  communications  with  finer  resolution  may  provide  further  insight 
into  interactions  within  complex  networks. 

This  section  identifies  parameters  of  complex  networks  and  the  established  meanings  from  the 
communication  network  and  social/cognitive  network  domains.  We  use  ELICIT  parameters  as 
the  common  thread  to  bridge  the  two  models.  With  the  over  50  parameters  of  the  sensemaking 
agent,  we  have  identified  a  subset  of  these  parameters  that  have  meaning  in  both  communication 
and  social/cognitive  domains.  The  objective  of  this  study  is  to  define  the  dual  use  of  the 
parameters  in  both  domains  to  allow  meaningful  experimentation  of  complex  networks.  Table  1 
provides  a  list  of  the  ELICIT  agent  parameters  and  their  meaning  in  both  the  communication  and 
social/cognitive  domains. 


Communication 

Social/Cognitive 

ELICIT 

Network 

Network 

Parameter 

Inbox  details 

Buffer  size 

Working  memory,  Memory 
capacity 

messageQueueCapacity 

TTL,  timeout 

Memory 

messageQueueTimeRemainlnQueue 

message  stack,  FIFO 

Message  reading  strategy 

messageQueueNewerBef oreOlder 

Message  Sending 

message  forwarding 

Share  before  reading 

shareBef ore Processing 

information  loss 

Broadcast  information 

postFactor 

information  loss 

Selective  forwarding 

shareWithFactor 

information  loss 

Information  seeking 

pullFactor 

information  sharing  rate 

Technology  preference 

SharingModality 

Delays 

message  preference 

screeningSelectedMessageDelay 

Encoding,  encryption, 

cognitive  processing  delay 

inf ormationProcessingDelay 

decoding 

(event-related  potential) 

Packet  formatting,  circuit 
establishment,  session 

Workload  (social) 

socialProcessingDelay 

Transmission,  post  latency 

Workload  (social) 

sharingPostingMessageDelay 

Local  topology  assessment 
(1-hop  neighbors) 

Workload  (cognitive) 

awarenessProcessingDelay 

Message  latency 

Human-computer  interaction 

webRequestDelay 

Availability,  connectivity 

Human-computer  interaction 

pul IBe tweenS ites Delay 

Human-computer  interaction 

pos tBe tweenS ites Del ay 

Memory  access 

Human-computer  interaction 

selectMessageFromQueueDelay 

Social  Processing 

Topology 

Organizational  structure 

accessibleAgents 

Topology 

Organizational  structure 

accessible Sites 

Message  control 

Curiosity,  Workload 

pr opens ityTo Seek 

Message  control 

Workload 

minTimeBetweenPulls 

Message  control 

Workload 

minTimeBetweenShares 

Trust  in  Network 

Trust  in  Individuals 

trustlnlndi  victuals 

Trust  in  Network 

Trust  in 

Information/T  echnology 

trust InWebSites 

Topology 

Mission  assignment 

Primary 

Topology 

Mission  assignment 

Secondary 

Innovation,  Exploration 

propens it yToShareExternal 

Table  1:  Relationships  between  Communication  Network  Variables,  Social  Cognitive 
Network  Variables  and  ELICIT  Parameters 

See  also  Appendix  1  for  a  complete  listing  of  the  ELICIT  2.5  sensemaking  agent  parameters  and 
the  baseline  values  used  for  this  effort. 

Wireless  Emulation  Laboratory  (WEL) 

The  Army  Research  Laboratory  has  developed  the  Wireless  Emulation  Laboratory  (WEL)  [29- 
30]  as  an  environment  to  test  algorithms  and  applications  for  mobile  ad  hoc  networks.  Emulation 
provides  a  more  realistic  environment  than  simulation  while  avoiding  most  of  the  difficulties  of 
full-scale  experimentation.  In  a  typical  Mobile  Ad-hoc  Network  (MANET)  emulation,  the 
communication  medium  is  abstracted  while  the  systems  themselves  are  implemented  using  real 
hardware  and  software.  Emulation  environments  are  advantageous  because  they  enable  a  higher 
fidelity  environment  in  which  there  will  be  actual  algorithms  and  hardware  variables  (CPU,  bus, 
driver  speeds)  not  present  in  simulation.  Furthermore  they  are  typically  viewed  as  the 
intermediary  grounds  between  simulation  and  field  experimentation  because  these  elements  will 
be  more  accurate  in  emulation  and  may  not  exist  in  simulation. 

The  WEL  makes  use  of  the  emulation  environment  Extendable  Mobile  Ad-hoc  Network 
Emulator  (EMANE)  [31].  In  this  environment,  components  of  the  tactical  environment  can  be 
configured  or  simulated/emulated  for  a  network  of  up  to  1000  nodes.  The  general  architecture 
and  modules  that  the  WEL  can  configure  are  shown  in  Figure  4.  These  modules  can  be 
configured  to  represent  specific  tactical  networking  scenarios.  For  instance,  given  terrain  data 
obtained  from  geographic  information  system  (GIS)  sources  such  as  Google  Earth,  Terrain 
Integrated  Rough  Earth  Model  (TIREM)  has  the  ability  to  predict  radio  frequency  (RF) 
propagation  patterns  based  on  the  terrain  models.  Software  called  Topodef  [32]  has  been 
developed  to  specify  topology  and  mobility  dynamics.  There  are  also  capabilities  allowing  for 
the  generation  of  simulated  network  traffic  to  determine  how  well  communication  networks 
handle  congestion.  The  EMANE  environment  also  allows  for  configurable  network  protocols 
such  as  the  routing  protocol,  Medium  Access  Control  (MAC)  schemes  and  other  security 
services.  WEL  robustly  models  communication  network  performance,  and  doesn’t  model  any 
intelligence  at  network  endpoints  so  it  is  a  perfect  complement  to  a  system  that  models 
intelligent  information  processing  at  network  endpoints. 
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Figure  4.  Overview  of  architecture  and  modules  of  the  Wireless  Emulation  Laboratory 
(WEL) 

Thus,  using  the  WEL  environment  with  ELICIT  may  provide  a  higher  fidelity  representation  of 
the  communications  typically  implemented  in  tactical  networks.  The  integration  of  ELICIT  into 
the  WEL  will  provide  a  means  to  run  experiments  with  higher  fidelity  communication  models  by 
using  elements  of  EMANE  such  as  the  routing  and  MAC  protocols.  As  seen  in  [26]  the  impact  of 
network  effects  on  the  task  performance  of  the  group  can  be  quantified  by  using  the  ELICIT 
agent  parameters  to  simulate  link  qualities  in  terms  of  loss  and  delay.  By  using  EMANE,  it  will 
be  possible  to  implement  link  quality  of  service  by  using  emulated  communications.  This  section 
describes  the  steps  taken  to  integrate  the  ELICIT  and  WEL  environments. 

Integrated  Communication  and  Social  Network  Model 

In  order  to  create  a  platform  that  models  both  communication  networks  and  social  networks, 
ELICIT  and  WEL  are  integrated.  The  integrated  environment  represents  communication 
networks  with  increased  fidelity.  The  integration  approach  places  a  configurable  ELICIT  agent 


on  each  WEL  node.  The  figure  below  depicts  an  ELICIT-WEL  integration  for  a  two  node 
system. 


Figure  5.  Integration  of  ELICIT  and  WEL  process  showing  communications  between  two 
nodes  and  the  ELICIT  server 

This  diagram  shows  the  communication  required  for  the  ELICIT  agent  on  EMANE  node  1  to 
share  a  factoid  with  the  ELICIT  agent  on  EMANE  node  2.  Each  EMANE  node  has  the  capacity 
to  be  both  a  sending  and  a  receiving  node.  Using  EMANE’s  capabilities  for  autoconfiguring 
EMANE  nodes,  this  two  node  configuration  is  instantiated  on  hundreds  of  nodes. 

In  ELICIT  version  2.4,  when  ELICIT  agent  1  shares  a  factoid  with  ELICIT  agent  2,  all 
communications  take  place  in  the  ELICIT  server  process  and  take  place  immediately.  EMANE 
has  a  sophisticated  configurable  process  for  modeling  communications  delays.  So  to  leverage 
this  capability,  EMANE  is  used  to  model  network  delays  between  ELICIT  agents.  The 
fundamental  approach  is  that  when  ELICIT  agent  1  shares  a  factoid  with  ELICIT  agent  2,  instead 
of  the  factoid  being  communicated  to  agent  2  immediately,  it  is  put  in  a  queue  on  the  ELICIT 
server  and  a  message  is  sent  via  a  shim  to  the  EMANE  node  on  which  agent  1  is  running.  Then 
EMANE  node  1  communicates  with  EMANE  node  2.  The  time  required  to  do  this  can  be  varied 
by  the  EMANE  system  to  model  various  types  of  communications  delays  and  outages,  etc.  Once 
the  shim  message  from  EMANE  node  1  is  communicated  to  EMANE  node  two,  EMANE  node 
2  sends  a  message  to  the  ELICIT  server  and  ELICT  releases  the  factoid  to  agent  2. 

Several  communication  network  parameters  in  EMANE  are  controlled  with  the  RF  Pipe  PHY 
model  parameters  [33],  Along  with  Topodef,  the  RF  pipe  physical  model  determines  the 
connectivity  of  the  network  throughout  the  experiments.  The  EMANE  experiment  scenario  is 
generated  using  the  Topodef  mobility  data,  which  calculates  path  loss  values  for  each  pair  of 
nodes  for  every  time  step.  The  presence  of  a  link  for  each  pair  of  nodes  is  determined  by  the 
following  expression  from  [33]: 

RxPower  (dBm]  =  Tx  Power  (dBm)  +  Tx  Antenna  Gain  (dBi)  +  Rx  Antenna  Gain  (dBi) 

-  Pathloss  (dB). 


The  RF  pipe  model  specifies  the  TX  Power,  Tx  Antenna  Gain  and  Rx  Antenna  Gain.  In  this 
work,  we  only  vary  TX  Power  and  leave  both  antenna  gains  to  be  0  dBi.  Additionally,  static 
topologies  are  used.  For  unsuccessful  transmissions  within  EMANE,  the  transmitting  node 
retransmits  10  times  before  giving  up.  This  behavior  was  adopted  in  the  shim  design  described  in 
the  Integration  Phase  3.  This  integration  approach  allows  us  to  leverage  the  full  modeling 
configurability  of  both  ELICIT  and  WEL.  It  required  enhancements  to  the  ELICIT  platform  and 
extension  of  WEL  via  ELICIT-WEL  shims.  This  integration  is  achieved  using  a  phased  approach 
which  is  summarized  below 

Integration  Phase  1 

•  Enabling  ELICIT  to  support  modeling  of  network  delays  and  links  outside  of  ELICIT. 
ELICIT  is  modified  so  that  it  can  be  configured  to  put  shared  factoids  in  a  queue  and  only 
release  them  when  the  release  is  specifically  requested  (after  a  modeled  delay.)  This 
functionality  is  tested  using  a  static  black  box  in  Phase  1 . 

Integration  Phase  2  (which  itself  consists  of  three  parts) 

•  Enabling  ELICIT  to  run  agents  on  separate  machines  so  that  ELICIT  can  be  used  in 
conjunction  with  WEL.  Previously  ELICIT  agents  could  only  run  on  the  same  machine  as 
the  ELICIT  server  process.  We  leverage  the  ability  of  humans  to  participate  in  an  ELICIT 
trial  via  a  browser  and  wrap  an  HTTP-client  around  the  agent  software  so  that  an  agent 
can  run  on  any  machine  connected  to  an  internet/  intranet  and  communicate  with  the 
ELICIT  server. 

•  Enabling  ELICIT  to  send  a  shim  message  to  the  EMANE  node  on  which  it  is  running. 
ELICIT  is  modified  so  that  after  an  agent  shares  a  factoid  (and  the  factoid  is  put  in  a 
queue  to  wait  for  the  modeled  delay)  the  agent  also  makes  a  POST  http  request  with  the 
following  parameters:  shareDate,  senderld,  recipientld  and  factoid  key  number.  Note  that 
shareDate  is  a  String  representation  of  the  Date  in  format:  MMddyyyNNmmssSSS, 
where 

o  yYear 
o  M  Month  in  year 
o  d  Day  in  month 
o  H  Hour  in  day  (0-23) 
o  m  Minute  in  hour 
o  s  Second  in  minute 
o  S  Millisecond 

This  shim  message  provides  all  the  information  needed  to  fully  describe  the  sharing  of  a 
particular  factoid  between  two  agents  and  communicate  that  to  EMANE. 

•  Enabling  ELICIT  to  release  the  shared  factoid  once  it  receives  the  appropriate  shim 
message  from  EMANE.  The  mechanism  created  in  phase  one  is  modified  so  that  a 
particular  factoid  is  released  to  the  receiving  agent  once  a  message  to  release  that 
particular  factoid  to  that  agent  is  received  by  the  ELICIT  server  from  EMANE.  A  utility 
called  update-share-delay.do  is  created  to  trigger  this  message  release.  The  following 
message  format  is  an  example  of  how  this  utility  is  used  to  tell  the  ELICIT  server  to 


release  factoid  17  that  was  shared  by  participant  2  to  participant  1: 
(http://util.azigo.net:8086/ccrp2.5_b7/update-share- 

delay.do?senderId=2&recipientId=l&factoidKey=17).  Thus  we  can  precisely  control 
when  a  receiving  participant  will  get  a  delayed  message. 

As  with  all  ELICIT  enhancements,  this  capability  is  configurable  and  can  be  turned  off. 
This  new  delay  behavior  is  only  operational  if  a  Network  Connectivity  Scenarios  option 
is  selected  for  an  ELICIT  trial.  This  Network  Connectivity  Scenario  file  (NCS 
Configuration  file)  in  turn  references  a  NCS  file  which  specifies  a  matrix  of  delays. 
Shared  factoids  are  only  held  in  a  queue  if  the  value  specified  in  the  NCS  file  is  greater 
than  zero. 

Integration  Phase  3  (which  itself  consists  of  three  parts) 

WEL  is  extended  with  a  software  shim  that  is  used  to  provide  communications  links  between 
ELICIT  and  EMANE  (same  EMANE  test  node).  There  are  three  aspects  to  this  shim: 

•  Creating  an  EMANE  shim  capability  that  can  accept  a  POST  http  request  from  the 
ELICIT  agent  on  a  EMANE  node  that  is  a  notification  that  an  Agent  has  initiated  a  share 

•  Creating  an  EMANE  to  EMANE  shim  capability  to  pass  this  request  to  the  receiving 
EMANE  node  via  EMANE  so  that  EMANE  can  be  leveraged  to  model  delays 

•  Creating  an  EMANE  shim  capability  to  update  the  ELICIT  server  to  indicate  that  a 
specific  factoid  had  been  communicated  via  EMANE.  Thus  a  shared  factoid  is  only 
released  to  the  receiving  participant  after  EMANE  has  directly  modeled  the 
communications  delay  between  the  sending  and  receiving  nodes. 

The  capabilities  developed  in  the  above  phases  are  integrated,  tested  and  refined  on  an  intranet  at 
ARL.  Once  the  new  ELICIT  capabilities  and  EMANE  shim  extensions  are  developed,  they  are 
deployed  into  the  WEL.  Scripts  are  developed  to  automatically  launch  an  emulation  scenario  for 
the  EMANE  test  nodes.  The  scripts  also  launch  the  ELICIT  agents  and  the  EMANE  shims, 
logging  the  agents  on  to  the  ELICIT  application  and  preparing  the  shims  to  accept 
communications  within  EMANE.  These  scripts  are  highly  configurable  to  accept  any  topology  or 
organizational  structure  as  well  as  different  ELICIT  agent  software  and  configurations  (different 
agent  personalities  for  each  agent  instantiation). 

To  further  automate  research  processes,  the  ability  to  configure  agent-based  ELICIT  experiments 
to  run  in  batch  mode,  is  extended  to  include  runs  using  distributed  agents  installed  on  EMANE 
nodes. 

Integrated  Platform  Experiments 

For  the  first  time,  we  now  have  a  configurable  operational  research  platform  that  models  the 
communication,  information  and  social/cognitive  aspects  of  complex  networks.  This  complex 
network  research  platform  leverages  proven  network  research  platforms:  WEL  and  ELICIT.  We 
are  now  in  a  position  to  perform  highly  automated  research  trials  that  much  more  fully  model  the 
behavior,  efficiency  and  effectiveness  of  human  network  operations. 


In  previous  work,  we  used  the  ELICIT  agent  parameters  to  represent  communication  network 
effects  to  determine  the  impact  on  task  performance  [26].  Prior  to  this  work,  the  communications 
models  within  ELICIT  were  idealized  with  respect  to  the  communication  network.  The  network 
did  not  incur  any  delays  or  drops  of  factoid  shares  or  interaction  with  websites.  By  implementing 
losses  and  delays  in  the  communications,  we  were  able  to  identify  the  impact  of  these  network 
parameters  on  the  task  performance  of  the  group.  The  result  of  these  experiments  was  the 
determination  that  information  delay  had  a  severe  threshold  effect  on  the  performance  of  the 
ELICIT  experiments.  Given  moderate  information  loss,  the  organization  was  able  to  overcome 
these  losses  due  to  redundancy  of  information  within  the  organizations.  Additionally, 
performance  was  studied  as  a  function  of  network  size  as  well  as  connectivity.  With  regard  to 
connectivity  the  agents  experienced  a  level  of  information  overload  in  situations  with  increasing 
network  connectivity.  The  results  were  simulation-based,  and  the  open  question  remained 
whether  or  not  organizations  would  actually  reflect  this  behavior  given  communications  models 
with  higher  fidelity. 

The  integration  of  the  ELICIT  and  WEL  platform  allows  for  a  comparative  validation  of  the 
parameter-based  simulations  of  the  communication  networks  within  the  ELICIT  experiments  in 
[26].  Based  on  the  previously  run  experiment,  we  conduct  a  similar  set  of  experiments  to 
determine  the  appropriate  modeling  of  the  ELICIT  experiments  when  using  the  agent  parameters 
to  represent  the  communication  models.  In  this  section,  we  will  present  results  of  the  ELICIT 
experiments  examining  three  different  network  parameters:  scalability,  information  loss  and 
delay,  and  connectivity.  If  differences  in  the  impact  of  these  network  parameters  are  observed, 
then  it  may  be  possible  to  determine  possible  shortcomings  of  the  model  and  will  allow  for 
adjustments  in  the  simulation  models  to  be  made.  Observed  differences  may  highlight  the  benefit 
of  the  increased  fidelity  of  the  communication  models  that  are  otherwise  difficult  to  represent 
easily  within  the  simulated  model. 

Scalability 

Understanding  the  impact  of  organization  or  network  size  is  crucial  to  the  modeling  of  complex 
networks.  We  ran  ELICIT  experiments  with  networks  of  n  =  {17,  34,  68,  100}  nodes.  Our 
initial  objective  of  this  set  of  experiments  was  to  determine  the  ability  of  the  ELICIT  platform  to 
support  larger  organizations  or  networks.  The  expectation  is  that  the  time  to  perform  the  task  will 
increase  as  a  function  of  the  network  size.  In  [26],  it  was  shown  that  the  ELICIT  platform  could 
be  run  on  networks  of  up  to  150  nodes.  The  scalability  of  the  communication  network  with 
respect  to  the  ELICIT  platform  can  be  determined  with  the  integrated  platform.  With  this  set  of 
experiments,  the  reduction  in  average  correctness  is  shown  as  a  function  of  time.  This  is  shown 
in  Figure  6.  The  general  behavior  of  the  average  correctness  in  these  experiments  is  similar  to 
those  found  in  [26], 


Figure  6.  Average  correctness  vs.  time  (hr)  for  network  sizes. 

In  addition  to  average  correctness,  it  is  of  interest  to  study  the  time  that  it  takes  any  node  within 
the  network  to  achieve  full  correctness.  It  may  only  be  required  for  a  single  decision-maker  to 
have  the  full  correctness.  We  ran  a  set  of  experiments  to  understand  how  the  time  it  takes  for  one 
node  in  the  network  to  achieve  full  correctness  as  a  function  of  the  network  size.  Figure  7  shows 
that  the  time  increases  exponentially  as  a  function  of  n. 
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Figure  7.  Time  (min)  required  for  first  node  to  achieve  full  correctness  vs.  n. 

Additionally,  we  can  look  to  the  number  of  successful  shares  executed  in  the  experiments.  When 
looking  at  this  property  with  respect  to  network  size,  we  expect  an  increase  in  shares  per  time 
interval  as  the  number  of  nodes  increases.  In  Figure  8,  we  show  the  total  number  of  shares  for  the 
first  30  minutes  of  the  experiment.  We  observe  that  there  is  a  drop  in  the  shares  after  the  68-node 
experiment.  While  one  would  expect  that  the  number  of  shares  to  increase  with  time,  the  effect 
of  the  overhead  of  the  communication  protocols  is  potentially  expressed.  Once  the  network  size 
is  larger  than  68  nodes,  the  network  may  be  experiencing  higher  congestion  and  collisions.  This 
is  an  item  for  future  verification  and  investigation.  Nonetheless,  it  appears  that  there  is  a 
performance  and  efficiency  tradeoff  with  respect  to  the  number  of  shares  as  the  network  size 
varies. 

isl 
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Figure  7.  Total  number  of  shares  in  a  30-minute  period  vs.  network  size  n. 

The  main  distinction  between  the  simulated  experiments  and  the  integrated  experiments  is  the 
underlying  communication  protocol  in  the  emulation  environment.  The  parameters  in  the 
simulated  experiments  implemented  the  loss  and  delay  between  two  nodes.  However,  in 
communication  networks,  most  communications  occur  over  multiple  hops,  where  the  network 
quality  of  service  may  vary  for  each  intermediate  link.  In  [26]  a  G(n,r)  network  was  applied  to 
create  the  connectivity  within  the  ELICIT  organization.  In  the  integrated  experiments,  we  are 
able  to  allow  each  of  the  nodes  to  share  with  each  of  the  other  nodes,  where  the  underlying 
communication  network  is  inherently  multi-hop.  This  is  one  example  of  the  increased  resolution 
of  the  integrated  environment. 

Connectivity 

In  this  section,  we  show  the  effect  that  connectivity  of  the  organization  has  on  the  correctness 
measure  within  ELICIT.  In  [26],  the  communication  radius  r  is  varied  when  creating  G(n,r)  to 
generate  networks  with  various  connectivity.  In  terms  of  the  communication  network,  the  cost  of 
increased  connectivity  is  increased  energy  consumption.  In  this  set  of  experiments,  we  created 
networks  of  34  nodes  and  varied  the  connectivity  by  adjusting  the  TX  Power  parameter  in  the 
RF  pipe  model.  TX  Power  =  40  resulted  in  no  links  being  formed,  and  TX  Power  =  60 
generates  a  fully  connected  network.  Figure  8  shows  the  average  correctness  after  30  minutes  as 
a  function  of  TX  Power.  A  sharp  threshold  with  regard  to  connectivity  is  exhibited.  This 
behavior  may  be  a  result  of  the  actual  topology  and  path  loss  models  being  applied  rather  than 
the  G(n,r)  model,  which  may  have  created  an  over  simplified  topology  representation. 


time  (hr) 

Figure  8.  Average  correctness  vs.  time  for  various  connectivity  with  n  =  34. 

In  [26],  increased  connectivity  resulted  in  the  agents  becoming  overwhelmed  with  the  factoids, 
not  allowing  them  to  process  the  factoids  necessary  to  achieve  full  correctness.  This  became 
more  prominent  as  connectivity  increased.  In  these  experiments,  we  observe  an  increase  in 
average  performance  as  connectivity  is  increased.  However,  for  larger  networks  (greater  than 
51),  in  situations  with  high  connectivity,  the  routing  protocol  is  overwhelmed  and  is  unable  to 
perform  the  ELICIT  experiment.  This  is  one  example  of  communication  network  layer  overload, 
which  is  similar  in  effect  to  social/cognitive  information  overload,  but  a  distinct  concept. 

Conclusions  &  Discussion 

We  now  have  an  operational  research  platform  that  can  simultaneously  model  both  the 
communications,  and  information  and  social/cognitive  aspects  of  complex  networks.  This 
enables  us  to  begin  to  explore  the  effects  of  communications  on  human  task  efficiency  and 
effectiveness  so  that  we  can  determine  how  best  to  optimize  the  use  of  available  bandwidth  to 
maximize  operational  effectiveness  under  varying  communications  conditions.  Agility  theory 
suggests  that  given  certain  network  conditions  (presence  of  network  effects),  parameters  within 
the  network  can  be  adapted  to  maintain  acceptable  levels  of  performance.  Building  operational 
models  to  characterize  and  explore  the  interactions  within  complex  networks  enables  the 
development  of  strategies  to  implement  agility  into  tactical  networks. 

The  use  of  communication  networks  yields  an  improved  representation  of  actual 
communications  protocols  as  compared  to  previous  simulations  and  experiment  setups.  The 
result  is  a  capability  that  can  directly  assess  relationships  between  communication  network  and 
social/cognitive  networks.  The  finer  resolution  of  communication  networks  allows  for  visibility 
into  the  routing  protocol,  traffic  congestion  and  transmission  collisions.  This  approach  also 
considers  the  effect  of  multi-hop  communications.  The  ability  to  model  these  concepts  will  help 
in  future  experiments.  This  paper  has  shown  preliminary  evidence  of  the  value  of  this  improved 
modeling  of  the  communication  models  within  the  larger  experiment  setup.  We  have  compared 
previous  simulation  results  with  experiments  conducted  in  the  integrated  ELICIT-EMANE 
platform.  In  the  future,  this  setup  will  allow  for  experiments  studying  specific  scenarios  such  as 
local  network  interruptions  due  to  environmental  effects  or  network  attacks. 
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Appendix  1  -  Agent  Parameters 


Below  is  a  sample  Agent  configuration  file,  with  initial  values  that  are  representative  of  the 
values  used  in  this  effort.  Note  that  time  intervals  are  in  milliseconds  and  this  configuration  is 
for  the  EBR  (2.5)  version  of  the  Sensemaking  Agent. 

1 .  readyIntervalDelay|Time  interval  to  click  Ready  button|  1 0000 

2.  messageQueueCapacity|Capacity  of  queue  (-1  means  unlimited)!- 1 

3.  messageQueueTimeRemainInQueue|Time  a  factoid  can  remain  in  queue  (-1  means 
unlimited)!- 1 

4.  messageQueueNewerBefore01der|If  true  then  newer  messages  are  selected  before 
older|false 

5.  selectMessageFromQueueDelay|Select  message  from  queue  delay)  1000 

6.  shareBeforeProcessing|If  true  then  share  message  before  Processing|false 

7.  postedTypes|PostedTypes|who, what, where, when 

8.  postF actor |PostF actor 1 1 

9.  postOutOfArea|PostOutOfArea|true 

10.  shareWithFactor|ShareWithF actor|  1 

11.  sharedTypes  |  SharedTypes|who, what, where, when 

12.  shareRelevantAccordingToSiteAccess|ShareRelevantAccordingToSiteAccess|false 

13.  shareAccordingToSiteAccess|ShareAccordingToSiteAccess|false 

14.  isCompetitiveHoarder|IsCompetitiveHoarder|false 

15.  pullFactor|PullFactor|0 

16.  timeBeforeFirstIdentify|Time  before  the  agent  does  its  first  identity  (in  minutes)!  15 

17.  minSolutionAreas|The  minimum  number  of  ID  tables  with  some  data|l 

18.  hasSeenEnoughToIdentify|HasSeenEnoughToIdentify|5 

19.  isGuesser|IsGuesser|true 

20.  isFrequentGuesser|IsFrequentGuesser|false 

21.  idConfidencelevel|IdConfidencelevel|0.23 

22.  partialldentify | Identify  if  there  are  no  some  answers|true 

23.  shareModalChoice|ShareModalChoice  possible  values  (both,  post  dominant,  post  only, 
peer  to  peer  dominant,  peer  to  peer  only)|peer  to  peer  only 

24.  screeningSelectedMessageDelay|Screening  selected  message  (message  processing) 
delay!  1000 

25.  informationProcessingDelay|Information  Processing  delay|3000 

26.  socialProcessingDelay|Social  Processing  delay|4000 

27.  sharingPostingMessageDelay|Sharing/Posting  each  Message  delay|8000 

28.  awarenessProcessingDelay|Awareness  Processing  delay|3000 

29.  determiningKnowledgeNeedsDelay|Determining  Knowledge  Needs  delay|3000 

30.  idAttemptDelay|ID  Attempt  delay|20000 

31.  webRequestDelay|Web  Request  (Pull)|9000 

32.  propensityToSeek|PropensityToSeek  possible  values  (low,  moderate,  high,  very 
high)  |  low 

33.  minTimeBetweenPulls|If  the  time  since  the  last  pull  is  not  >=  minTimeBetweenPulls,  do 
not  Pull  (in  milliseconds,  -1  means  ignoring  this  parameter)|20000 


34.  minTimeBetweenShares|If  the  time  since  the  last  Share  is  not  >= 

minTimeBetweenShares,  the  agent  should  wait  before  it  Shares  (in  milliseconds,  -1 
means  ignoring  this  parameter)|5000 

35.  trustInIndividuals|TrustInIndividuals  possible  values  (high,  medium,  distrust,  no 
opinion)| l=no  opinion, 2=no  opinion, 3=no  opinion, 4=no  opinion, 5=no  opinion, 6=no 
opinion, 7=no  opinion, 8=no  opinion, 9=no  opinion,  10=no  opinion,  1  l=no  opinion,  12=no 
opinion,  13=no  opinion, 14=no  opinion, 15=no  opinion, 16=no  opinion, 17=no  opinion 

36.  trustInWebSites|List  of  initial  values  of  Trust  for  web  sites.  Possible  values  (high, 

medium,  distrust,  no 

opinion)|who=medium,where=medium,what=medium,when=medium 

37.  reciprocity|Reciprocity  possible  values  (high,  low,  medium,  na, 

none)|l=none,2=none,3=none,4=none,5=none,6=none,7=none,8=none,  9=none, 

1 0=none,  1 1  =none,  1 2=none,  1 3=none,  1 4=none,  1 5=none,  1 6=none 

38.  primary |Primary  areas  of  interest.  Possible  values:  who,  what,  where, 

when)  |  who,  what, when,  where 

39.  secondary | Secondary  areas  of  interest.  Possible  values:  who,  what,  where,  when)| 

40.  propensityToShareExternal|If  message  is  not  in  area  of  interest,  then  agent  shares  it 
according  to  sharing  preferences  with  probability  =  propensityToShareExtemal|l 

41.  awarenessProcessingThreshold|If  cumulative  value  of  the  perceived  message  value  is 
more  or  equal  to  this  variable,  then  start  awareness  processing. |2 

42.  pullBetweenSitesDelay|Pull  between  sites  delay]  1000 

43.  postBetweenSitesDelay|Post  between  sites  delay|500 

44.  provideRelevance (Provide  relevance  for  posted  and  shared  messages|false 

45.  provideTrust|Provide  trust  for  posted  and  shared  messages|false 

46.  sharingModality|peer  to  peer  only,  post  only,  both|peer  to  peer  only 

47.  shareValueThreshold|Factoid  value  threshold  for  sharing  and  posting,  integer  value  or 
high,  medium,  non-zero,  none=any|none 

48.  shareTrustThreshold|Factoid  trust  threshold  for  sharing  and  posting,  high,  medium,  no 
opinion,  distrust=none=any|none 

49.  shareSourceThreshold|Factoid  direct  source  threshold  for  sharing  and  posting,  high, 
medium,  no  opinion,  distrust=none=any|dist 

50.  postBeforeProcessing|Whether  agent  should  post  immediately  before  determining  factoid 
value|false 

51.  shareBeforePost|Whether  agent  should  do  shares  or  posts  first|true 

52.  noSharingIfPosted|In  “post  before  share”,  whether  agent  should  take  success/failure  of 
posts  into  account  for  sharing,  no,  any  or  all|no 

53.  accessible  Agents  |  Agents  that  can  be  shared  with,  list  of  agents,  or  all  or  none|all 

54.  accessibleSites|Sites  that  can  be  posted  to  and  pulled  from,  list  of  sites  or  all,  none, 
primary  |  all 
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0  RDECOM  jr  Motivation 


•  Tactical  networks  are  comprised  of  many  dynamic  elements  that  focus  on 
providing  support  to  the  Warfighter 

•  Need  to  understand,  model,  predict  and  analyze  Warfighter  performance  in  a 
complex  environment: 

-  i.e.  Military  assets,  physical  environment,  social  structures 
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Approaches:  Network  Science 


Network  science 

-  Understand  the  dynamics  and  relationships 
between  complex  network  layers 

-  Enhance  human  performance  for  network- 
enabled  warfare 

-  Enhance  speed  and  precision  for  complex 
military  operations. 
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0  RDECOM  ?  Approaches:  C2  Maturity 


Command  and  Control  Maturity,  Agility 

-  Complex  endeavors 

-  Organizations  operate  within  the  C2  maturity  space 

•  distribution  of  information,  patterns  of  interaction,  allocation  of  decision  rights 

-  C2  Agility 

•  robustness  of  operating  conditions,  adaptation  of  varying  conditions 
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Goal:  Develop  experimental  capability  that  address  difficulties/shortcomings 
for  complex  networks 


Current  lack  of  models  for  complex  networks 
Difficulties: 

-  Meaningful  variation  of  network  effects 

-  Lack  of  inclusion  of  multiple  complex 
network  layers 

-  Measuring  human  performance 
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*)  RDECOM /  Modeling  Complex  Networks 


Ability  to  differentiate  between  types  of  delays  to  further  increase  the  fidelity  of 
the  simulation 

-  Communication  network  effects  vs.  cognitive  /  social  effects 
Leverage  existing  experiment  tools  (i.e.  ELICIT  part  of  the  complex  network) 

-  Add  representation  for  other  layers 
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0  RDECOM  J  Experiment  Platform 


A 


•  ELICIT 

-  Experimental  Laboratory  for  Investigating  Collaboration,  Information-sharing 
and  Trust 

-  Sponsored  by  CCRP  to  test  principles  of  C2  /  organizational  theory 

-  Initially  designed  for  people  to  participate  in  an  exercise  to  identify  a  fictional 
terrorist  threat 

-  Modified  to  enable  sensemaking  agents  run  in  place  of  people 
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The  ELICIT  fictitious  terrorism  plot:  situational  awareness  is  measured  by  the 
claimed  fraction  of  information  about  the  terrorist  threat 


Correctness  C  =0.25 (WHO  +  WHERE  +  WHAT  +  WHEN) 
Average  and  Maximum  correctness  is  considered: 

C(t)  =  ^  J2Ci(t)  maxC(f)  =  max  Ci(t)  V  i 

Time  first  agent  arrives  at  full  correctness  (timeliness) 
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Communication  Network 
Modeling  in  ELICIT 


A 


•  Represent  communication  network  properties 
using  ELICIT  sensemaking  agent  parameters 

-  scalability 

-  connectivity 

-  communication  loss  and  delay 


Communication  Network 

Sensemaking  Agent 

Probability  of  a  successfully 

ShareWithFactor 

transmitted  packet 

Packet  latency 

SharingPostingMessageDelay 

Network  topology 

Organization 

K.  Chan,  N.  Ivanic,  “Connections  between  communications  and  social  networks  using  ELICIT,”  ICCRTS  2010,  Santa  Monica,  CA,  June,  2010. 
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Multidisciplinary  Network 
Concepts 


Meaning  of  similar  parameters  across  complex  network  layers 
-  Rosetta  Stone  between  complex  network  layers 
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0  RDECOM  J  Wireless  Emulation  Laboratory 


•  WEL:  a  controlled,  repeatable 
emulation  environment  for  tactical 
wireless  Mobile  Ad  hoc  Networks 
(MANETs)  and  Information 
Assurance  (IA)  experiments 

-  EMANE 

•  up  to  600  virtual  nodes 

•  runs  actual  communication 
radio  code,  routing 
protocols,  medium  access 
control  protocols 

•  transmission  medium  is 
emulated 

-  Topodef:  specifies  topology 
and  mobility  over  duration  of 
experiment 
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Network 
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Making 


Scenario 


MANET 

Node 


Analysis 


Scenario 


Generator 
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ELICIT/EMANE  integration 


Integration:  Adding  the  EMANE  communication  network  models  into  ELICIT 


C2/0rganizational  theory 


Decision-making  scenario 


Sensemaking  agents 


Communication  network  models 
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ELICIT  Extensions 


ELICIT  enhancements  for  EMANE  integration  are  generally  available  in  ELICIT  2.5 

•  Support  for  distributed  agents  (agents  can  access  ELICIT  via  http,  just  as 
humans  do) 

•  EMANE  integration  compatibility 

-  Shim  communications  with  EMANE 

-  Shares  can  be  delayed  until  explicitly  released 

•  Can  schedule  human/distributed  agent  runs  in  batch  mode 

•  Additional  source  information  recorded  for  agent  use  in  trust  calculations 
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Experiments  conducted  using  the  integrated  ELICIT/EMANE  platform 

Experiments  similar  to  previous  work  using  sensemaking  agent  parameters  as 
the  communication  network  parameters 

-  Scalability:  n  ELICIT  nodes,  n  EMANE  nodes 

-  Connectivity:  G(n,  r)  -  transmit  power  (TX  power)  controlled  by  EMANE  RF 
pipe  physical  model 
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Integrated  Platform 
Experiment  Results:  Scalability 


Average  correctness  vs.  time  for  n  =  {17,  34,  51,  68, 100} 

Expected/validated  degradation  of  average  correctness  as  network  size 
increases 

Exponential  increase  of  time  required  for  first  full  correctness  as  n  increases 


Avg  Correctness  vs.  Time  (min) 


10  20  30  40  50  60  70 

network  size  (n) 
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Integrated  Platform 
Experiment  Results:  Connectivity 


•  Threshold  effect  of  performance  once 
connectivity  is  established 

•  Observation:  routing  overhead  caused 
nodes  to  be  unresponsive  at  moderate  to 
high  link  density.  Network  becomes 
overloaded 
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Increasing  TX  power,  connectivity 
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Conclusion 


•  Established  an  integrated  ELI  C  IT/EM  AN  E  experiment  platform  that  can 
represent  both  communication  network  and  social/cognitive  network  effects 

-  Preliminary  experimental  results  match  the  previous  work  using  ELICIT 
agent  parameters 

-  Integrated  model  demonstrates  existence  of  phenomena  similar  to  network 
overload  and  cognitive  overload 

-  Integrated  platform  enables  simultaneous  experimentation  of  both 
communication  and  social/cognitive  networks 
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Thank  You 


Questions? 
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